NPSOR-92-005 

NAVAL  POSTGRADUATE  SCHOOL 


Monterey,  California 


*  Approved  for  public  release;  distribution  is  unlimited.: 
Prepared  for: 

Naval  Weapons  Support  Center 

•  Crane,  IN  47522 


92-03309 


92  2  10  061 


TfflS  DOCUMENT  IS  BEST 
QUALITY  AVAILABLE.  THE  COPY 
FURNISHED  TO  DTIC  CONTAINED 
A  SIGNOTCANT  NUMBER  OF 
PAGES  WHICH  DO  NOT 
REPRODUCE  LEGIBLY. 


NAVAL  POSTGRADUATE  SCHOOL, 
MONTEREY,  CALIFORNIA 


Rear  Admiral  R.  W.  West,  Jr.  Harrison  Shull 

Superintendent  Provost 


This  report  was  prepared  for  and  funded  by  the  Naval  Weapons  Support  Center, 
Crane,  Indiana. 

This  report  was  prepared  by: 


Assistant  Professor, 

Department  of  Operations  Research 


Department  of  Operations  Research 


Reviewed  by: 


Released  by: 


Professor  and  Chairman 
Department  of  Operations  Research 


PAULJ.MARTO 
Dean  of  Research 


UNCLASSIFIED _ 

SECURITY  CLASSIFIC.-\TION  OF  THIS  PAGE 


la.  REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 


2a.  SECURITY  CUSSIFICATION  AUTHORITY 


2b.  DECUSSIFICATION/DOWNGRADING  SCHEDULE 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

NPSOR-92-005 


REPORT  DOCUMENTATION  PAGE 


lb.  restrictive  MARKINGS 


Form  Approved 
OMBNo.  0704-0188 


3.  DISTRIBUTION  /AVAIUBILITY  OF  REPORT 

Approved  for  public  release;  distribution  is 
unlimited. 


5.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a.  NAME  OF  PERFORMING  ORGANIZATION  6b.  OFFICE  SYMBOL  7a.  NAME  OF  MONITORING  ORGANIZATION 

{ifa^icabiei)  Naval  Postgraduate  School 

Naval  Postgraduate  School  OR 


8b.  OFFICE  SYMBOL 
{If  appltcabld) 


6c.  ADDRESS  {Cify,  State,  and  ZIP  Codd) 

Monterey,  CA  93943 


8a.  NAME  OF  FUNDING^SPONSORING 
ORGANIZATION 

Naval  Weapons  Support 
Center 


8c.  ADDRESS  (C//y,  State,  and  ZIP  Code) 

NWSC,  Crane,  IN  47522 


11.  TITLE  {Include  Secunty  Classification) 

Pyrotechnic  Device  Reliability 


12.  PERSONAL  AUTHOR(S) 


Lyn  R.  Whitaker  and  Michael  P.  Baile 


13a.  TYPE  OF  REPORT  13b  TIME  COVERED 

Technical  Report  from _ to _ 


16.  SUPPLEMENTARY  NOTATION 


COSATI  CODES 


7b.  ADDRESS  ( City,  State,  and  ZIP  Codd) 

Monterey,  CA  93943-5006 


9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


N4802991WXPM454 


10.  SOURCE  OF  FUNDING  NUMBERS 


i  PROGRAM 

PROJECT 

TASK 

WORK  UNIT 

j  ELEMENT  NO. 

NO. 

NO 

ACCESSION  NO. 

14  DATE  OF  REPORT  ( Year,  month  day)  15.  PAGE  COUNT 

December,  1991  25 


18  SUBJECT  TERMS  {Continue  on  reverse  if  necessary  and  identify  by  block  number) 
FIELD  I  GROUP  I  SUB-GROUP  |  Reliability,  pyrotechnic  devices 


19  abstract  {Continue  on  reverse  if  necessary  and  identify  by  block  number) 


This  report  looks  at  the  problem  of  determining  when  to  award  bonuses  for  reliability 
improvement  in  pyrotechnic  devices  based  on  data  gathered  under  a  lot  acceptance  sampling 
plan. 


^  0IS1 RIBUTION/AVAIUBILITY  OF  ABSTRACT 
■V  UNCUSSIFIED/UNLIMITED  H  SAME  AS  RPT 


22a  NAME  OF  RESPONSIBLE  INDIVIDUAL 

L.  Whitaker 


DD  Form  1473,  JUN  86 


DTIC  USERS 


21  ABSTRACT  SECURITY  CU3SICIATION 

UNCLASSIFIED 


22b  TELEPHONE  {Include  Area  Code)  2c  OFFICE  SYMBOL 

(408)  646-3482  OR/Wh 


Previous  editions  are  obsolete  SECURITY  CLASSIFICATION  OF  THIS  PAGE 

S/N  u:02-LF-014-6603  UNCLASSIFIED 


I 


Pyrotechnic  Device  Reliability 


Lyn  R.  Whitaker  and  Michael  P.  Bailey 
Department  of  Operations  Research 
Naval  Postgraduate  Scliool 
Monterey,  CA  93943 
(408)646-3482 
December  19,  1991 

Abstract 

This  report  looks  at  the  problem  of  determining  when  to  award  bonuses  for  reliability 
improwment  in  pyrotechnic  devices  based  on  data  gathered  under  a  lot  acceptance 
sampling  plan. 

1  Introduction 

As  government  procurement  contracts  are  now  written,  contractors  have  no  incentive  to 
improve  the  quality  of  the  items  they  provide.  To  improve  quality,  the  Xa\'al  Weapons 
Support  Center,  Crane,  IN,  has  decided  to  implemen*  a  bonus  system.  If,  as  a  result  of 
the  sampling  inspection  plan  currently  required  for  lot  acceptance,  there  is  e\idence  that 
the  lot  has  a  reliability  that  exceeds  the  minimum  requirement  for  lot  acceptance,  then  the 
contrarfor  will  be  eligible  for  a  bonus.  The  diffinilty  with  implementing  such  a  system  of 
bonuses  is  that  the  sampling  inspection  plans  are  designed  to  provide  clear  cut  criteria  for 
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lot  acceptance  or  rejections.  They  do  not  readily  provide  a  single  measure  of  quality  or 
reliability  of  the  item  being  tested. 

The  first  item  for  which  a  bonus  system  will  be  implemented  is  a  pyrotechnic  device. 
The  sampling  inspection  plan  for  this  detice  involves  sampling  items  from  a  lot,  subjecting 
them  to  a  manufacturing  test  and  exposing  them  to  one  of  three  environments.  An  attempt 
is  then  made  to  acti\'atc  each  device.  Specifically,  a  lot  of  500  to  1000  devices  is  accepted  if 
all  of  the  following  criteria  are  met: 

1.  Of  20  items  subjected  to  the  manufacturing  tost,  no  more  than  1  can  fail  to  activate. 

2.  Of  20  items  subjected  to  the  joim,  rnsnufacturing  and  temperature  and  humidity  test 
no  more  than  1  can  fall  to  activate, 

3.  Of  32  items  subjected  to  the  joint  manufacturing  and  vibration  test  no  more  than  2 
can  fail  to  artiwte, 

4.  Of  20  items  subjected  to  the  joint  manufacturing  and  altitude  test  no  more  than  I 
can  fail  to  acti\’ate. 

The  samples  in  tests  1-4  are  distinct  .  thus  a  total. of  !)2  items  are  tested. 

Attempts  have  been  made  to  estimate  the  reliability  /?  of  a  device  (here  reliability  is 
defined  as  the  probability  that  an  item  would  activate  after  the  manufacturing  test  and 
exposure  to  all  three  environments)  ba.sed  on  data  from  the  .sampling  inspection  plan  ([4] 
and  [5])  ,  The  methods  used  to  produce  lower  confidence  bo\mds  T.CR  for  this  reliability 
based  on  data  from  a  lot  which  is  just  barely  accepted  have  yeilded  values  that  are  much 
lower  than  observed  in  the  field. 

The  most  obvious  reason  for  this  discrepancy  is  that  requiring  successful  activation  after 
exposure  to  all  environments  is  much  more  stringent  than  requiring  a  device  off  the  shelf 


to  activate  in  practice.  An  operational  environment  would  not  include  all  these  adverse 
conditions.  It  is  also  likely  that  items  which  acti\ate  after  exposure  to  one  environment 
wotild  be  more  likely  to  activate  after  exposure  to  other  environments.  Unfortunately  the 
presence  or  absence  of  such  dependence  cannot  be  inferred  from  the  data  yielded  by  the 
current  sampling  iaspection  plan.  Thus,  the  models  that  have  been  used  to  estimate  the 
reliability  have  assumed  independence  of  the  events  that  a  device  activates  after  exposure  to 
the  three  cmironments  given  that  it  wuld  have  acti\ated  after  the  manufacturint^  test.  If 
there  is  positive  dependence  between  these  events  then  estimation  under  the  resumption  of 
independence  yields  reliabilities  and  LCB’s  that  are  lower  than  they  would  be  if  dependence 
were  accounted  for.  It  <»hnuld  be  noted  that  even  if  exact  LCB’s  can  be  found,  if  the 
modeling  assumptions  are  changed  to  include  dependence  (where  these  new  assumptions 
are  supported  by  empirical  evidence),  and  if  the  ’true’  reliability  could  be  defined  and 
estimated  from  the  available  data,  the  LCR’s  could  still  be  rather  low  for  the  simple  reason 
that  the  sample  sizes  of  the  sampling  inspection  plan  are  too  small  to  give  a  sharper  lower 
confidence  hound. 

In  Section  2.  we  find  the  maximum  liklihood  estimator  (.NILE)  of  R  under  the  assump¬ 
tion  of  independence  of  the  events  that  a  device  actitutes  after  exposure  to  the  three  en¬ 
vironments  given  that  it  has  pa.ssed  the  manufacturing  test.;  In  the  process,  we  uncover  a 
difiiculty  with  estimating  R  under  this  assumption.  Specifically,  this  procedure  can  yield 
lower  estimates  of  R  for  samples  which  have  fewer  failures.  To  fix  this,  lower  bounds  for 
the  MLE  of  R  under  the  conditional  independence  assumption  are  derived  and  shown  to 
be  lower  bounds  for  MLE’s  under  weaker  and  more  realistic  assumptions.  Approximate 
959(  LCB's  for  R  and  the  MLE  arc  provided  explicitly  for  the  24  realizations  of  the  sam¬ 
pling  plan  leading  to  lot  Mcceptance.  In  Section  :l  a  direct  approach  to  awarding  bonuses 


is  taken.  It  provides  an  alternative  to  using  LCB’s  for  decision  making.  Conclusions  and 
recommendations  are  given  in  Section  4. 

2  Estimating  Reliability 

Tests  2.  3,  and  4  in  the  lot  acceptance  sampling  plan  inr'^he  trying  to  activate  the  device 
after  it  is  subjected  to  the  manufacturing  test  and  one  of  the  environmental  tests.  To 
distinguish  between  the  outcome  of  these  joint  tests  and  what  would  have  happened  had 
the  device  been  subjected  to  one  of  the  tests  alone,  let  £'i,£2,£3,  and  £4  be  the  respective 
events  that  a  device  can  activate  after  being  subjected  to  just  the  manufacturing  test,  the 
temperature  and  humidity  test,  the  vibration  test,  and  the  altitude  test.  Then,  Jii  s  P(Ei) 
and  /?,  5  P(Ei  n  £,),  i  «  2.3,4  are  the  probabilities  that  a  device  activates  after  being 
subjected  to  tests  1,  2,  3,  or  4,  respectively..  The  reliability  of  a  device  is  defined  as 

R  =  P(£i  n  £2  n  £3  n  £4). 

H  will  be  estimated  based  on  X„  the  number  of  devices  which  activate  out  of  the  Uj 
that  are  s",bjected  to  test  I  The  sampling  plan  uses  difierent  devices  randomly  chosen  from 

the  lot  in  question.  Thus,  it  is  reasonable  to  assume  that  . .,Y4  are  independent  and 

X,  --  Binomial{n„  R,).  i  =  1 . 4. 


2.1  Maximum  Likelihood  Estimation 


The  likelihood  of  o!)sorving  X\  =  xi, _ ^4  =  X4  is  given  as 
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where  0  <  ffj  <  1,0  <  /?,  <  Wj,  /  =  2,3.4.  For  the  sampling  plan  described  in  the  intro¬ 
duction,  there  are  24  realizations  of  ,  ^4  that  lead  to  lot  acceptance.  The  maximum 
likelihood  estimates  (MLE’s)  Rt  of  Ri  for  these  24  realizations  are  given  in  Table  1. 

Because  the  same  device  is  never  actually  subjected  to  more  than  one  of  the  environmen¬ 
tal  tests,  the  estimators  Ri  of  if,  alone  are  not  enough  to  construct  an  MLE  for  R.  In  the 
subsequent  subsections,  we  describe  an  estimator  Rci  for  R.  which  is  baaed  on  the  assump¬ 
tion  of  conditional  independence.  After  examining  the  shortcomings  of  Rcj,  we  provide  a 
second  estimate  R^g  which  requires  no  assumptions  concerning  dependencies  between  test 
outcomes,  and  which  provides  a  lower  bound  for  R.. 

2,2  Conditional  Independence 

Additional  assumptions  about  the  dependence  between  the  events  £1 ....  £*4  are  needed 
to  estimate  R  based  on  the  current  sampling  plan  data.  The  most  direct  approach  for 
Mtimating  R  based  on  this  data  is  to  assume  that,  given  Fi.  that  To.  £3,  £4  are  independent 
events,  (see  [4]  and  [.’»]  for  further  discussion.)  The  MLE’s  ir.  Table  1  are  also  the  MLE’s  for 
Ri  under  the  assumption  of  cojtditional  independence.  With  this  assumption,  when  Ri  >  0, 
we  have 

R  —  £(£i  n  £2  n  £3  n  £4) 

=  £(£2  n  £30  £4  !£,)£(£,) 

=  £(£2|£i)P(£3|£i)P(£4|£i)£(£i) 

_  RoR^R^ 

-  ~W~’ 

Thus  the  MLE  Rcj  of  R  under  this  assumption  is 
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Ikble  1:  MLEs  for  iZi, ....  /Z4  computed  for  the  24  realizations  of  the  sampling  plan  leading 
to  lot  acceptance,  where  (i,  kt  1)  represents  the  number  of  failures  in  tests  1, . . . ,  4. 
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(0011) 
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(1010) 
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0.983 

0.969 

0.983 
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0.986 

0.950 

0.986 

0.986 

(0020) 

1.000 

1.000 

0.938 

1.000 

(0111) 

1.000 

0.950 

0.969 

0.950 

(lOU) 

0.975 

0.975 

0.969 

0.950 

(1101) 

0.981 

0.950 

0.981 

0.950 

(1110) 

0.975 

0.950 

0.969 

0.975 

(0021) 

1.000 

1.000 

0.938 

0.950 

(0121) 

1.000 

0.950 

0.938 

1.000 

(1020) 

0.983 

0.983 

■  0.938 

0.983 

(1111) 

0.962 

0.950 

0.961 

0.550 

(0121) 

1.000 

0.950 

0.938 

0.950 

(1021) 

0.975 

0.975 

0.938 

0.950 

(1120) 

0.975 

0.950 

0.938 

0.975 

(1121) 

0.950 

0.950 

0.938 

0.950 

fi 


2. 


>  0 

Rci  =  { 

[  0  hi  =  o. 

Realizations  of  Rci  for  the  24  outcomes  that  lead  to  lot  acceptance  are  given  in  Table 

From  Table  2  it  is  clear  that  the  assumption  of  conditional  independence  is  unacceptable. 
Under  conditional  indepence,  there  are  several  cases  where  sampling  plans  resulting  in  fewer 
failures  give  smaller  estimated  xetlue  of  R  than  with  more  failures.  An  example  of  this 
is  the  failure  configuration  (1, 1.1,0)  with  Ra  =  944  compared  to  the  case  (0,1, 1.0) 
with  Rci  =  .920.  What  happens  in  the  (0. 1. 1,0)  rase  is  that  the  ronditinnal  indepenoe 
asstjmption  will  attribtite  a  disproportionate  amount  of  the  ratise  of  faihire  to  the  lot’s 
inability  to  pass  environmental  tests  2  and  3.  Thus,  the  estimated  probability  of  passing  all 
tests  is  downgraded  too  much.  This  contradiction  will  lead  us  to  dispose  of  the  conditional 
independence  assumption  altogether.  We  will  now  make  the  results  of  this  phenomenon 
formal. 

Data  from  the  lot  acceptance  sampling  plan  can  be  thought  of  as  censored.  For  tests  2*  4, 
it  is  unknown  whether  failure  is  due  to  the  manufacturing  test,  the  particular  environmental 
test,  or  both.  This  lack  of  information,  along  with  the  conditional  independence  assumption, 
causes  the  apparent  reversals  in  Rci  observed  in  Table  2.  To  see  this,  suppose  that  we  could 
determine  the  cause  of  failure  for  a  particular  device  which  fails  to  acti\'ate.  For  test  i,  let 

•  n,  -  X,  be  the  total  number  of  failures  for  test  i\ 

•  Xi,i  be  the  number  of  failures  resulting  from  only  the  environmental  exposure  (these 
would  have  passed  the  simple  manufacturing  test); 

•  j,.2  be  the  number  of  failures  resulting  from  only  the  manufacturing  test,  (these  would 
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Table  2:  The  MLE  for  A  and  an  apprcodmate  95%  LCB  computed  for  the  24  realizations 
of  the  sampling  plan  that  lead  to  lot  acceptance,  where  (t,  j,  k,  1)  represent  the  number  of 
failures  in  tests  M  respectively 


ftcr 

95%  LCB 

(0000) 

1.000 

1.000 

(1000) 

0.989 

.900 

(0010) 

0.969 

.906 

aoio) 

0.969 

.855 

(0001) 

.950 

.850 

(0100) 

0.950 

.850 

(1001) 

0.950 

.828 

(1100) 

0.950 

.926 

(1011) 

0.944 

.802 

(1110) 

0.944 

.800 

(1111) 

0.939 

.765 

(0020) 

0.938 

.875 

(1020) 

0.937 

.813 

(1121) 

0.937 

.751 

(0011) 

0.920 

.823 

(0110) 

0.920 

.823 

(1101) 

0.920 

.775 

(1021) 

0.913 

.765 

(1120) 

0.913 

.761 

(0101) 

0.902 

.800 

(0021) 

0.891 

.775 

(0120) 

0.891 

.788 

(0111) 

0.874 

.750 

(0121) 

0.846 

.717 

have  passed  the  simple  environmental  test); 


•  Xi,3  be  the  number  of  failures  resulting  from  both  the  manufacturing  and  the  environ¬ 
mental  tests. 

The  likelihood  L  is  then  proportional  to 

\ 

II  P(£\  n  EiY'PiE^  n  P{EI  n  e)^'-^p{e\  n 
(=2 

Let  n  ss  n,.  Conditioning  on  the  event  E\  gives 

n  P{Ei\E^y'{\  -  p{E,\E^)y^^p{E^E\y^■^{\  -  P{E,\E{)y-^\: 

i“2 

Therefore,  the  MLE's  of  interest  are 
P(C,)  =  SkiLliit 

P(£,t£,)  =  /or/ =  2. 3.1. 

With  these  MLE’s  in  hand,  we  can  explain  the  behavior  of  the  estimates  in  the  condi¬ 
tional  independence  case.  When  £2.  £3,  £4  are  independent  given  £1,  we  have 

fia  =  ?(£■,)  nP(£.|£i) 

1=2 

E4  I  4 

n  Xi  +  XU  ‘ 


When  the  are  not  observed,  the  maximum  likelihood  approach  assigns  values 


X|,l 


^  P(£fn£0 

Pi E,n  Ex) 


f) 


X, 


Ri-R, 

Ri 


to  Xi.1  where  Ri  are  found  by  maximizing  (1).  This  is  what  leads  to  the  seemingly  non> 
sensical  ordering  of  reliabilities  seen  in  table  2.1.  The  worst  case  was  failure  configuration 
(1, 1.2, 1),  for  which  we  can  compute 


■^2.1  =  -h.i  = 


.937.1 
as  0,01333. 


Alternatively,  configuration  (0.1,2. 1)  results  in 

as  1 

.937."), 

as  2 

=  '»(^) 

»  1 


Thus,  the  second  configuration,  although  being  clearly  superior  to  the  first,  results  in  assign¬ 
ing  blame  to  the  environments  as  the  cause  of  failure.  Since  our  definition  of  reliability  is 
the  joint  probability  of  activating  under  all  circumstances,  the  failure  configurations  which 
appear  to  point  to  the  inann fart ii ring  test  as  the  cause  of  most  of  the  failures  artnally  yield 
greater  reliabilities. 
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Allowing  the  maximum  likelihood  procedure  to  assign  \’alues  to  the  censored  observar 
tions  leads  to  an  ordering  of  the  estimates  of  R  that  are  counterintuitive.  In  the  next 
section,  we  will  develop  a  bound  for  R  which  doesn’t  use  conditional  independence,  and 
which  produces  an  ordering  of  estimates  of  R  which  is  more  believable. 


2.3  A  Reasonable  Lower  Bound 

An  alternative  approach  is  to  note  that  no  matter  what  the  censored  observations  are.  we 
still  have 

/>(£•)  >  Skfi 

tt 

P(£il£i)  >  --/  =  2.3.4. 


Thus,  under  the  assumption  of  conditional  independence 


Rci  >  (- 


')n 


Note  that  the  lower  bound  of  Rci- 


(2) 


is  also  equal  to  the  NILE  when  there  is  as  much  dependence  between  Ei . . . ,  £4  as  is  possible. 
This  maximum  dependence  case  arises  if  we  assume  that  all  of  the  failures  observed  are  the 
result  of  BOTH  the  manfacturor’s  test  and  and  environmental  test.  i.e.  ,r,„3  =  n,-  -  or,-,  i  = 


2.3,4. 

Rlb  for  the  failure  configurations  leading  to  lot  acceptance  are  shown  in  Table  3.  There 
are  two  adxTintages  to  using  Rig  rather  than  Rci  to  estimate  R.,  First,  estimates  Rig  are 
ordered  in  a  reasonable  way.  failure  configurations  with  more  failures  always  have  lower 


Table  3:  The  lower  bound  for  the  MLE  of  R  computed  for  the  24  realizations  of  the  sampling 
plan  leading  to  lot  aoceptanne  where  represent  the  number  of  failures  in  teste  1-4 

respectively. 


kt,B 

(OOOQ) 

1.000 

(1000) 

0.9S9 

(0010) 

0.958 

(1010) 

0.948 

(GOOD 

0.940 

(0100) 

0.940 

(1001) 

0.938 

(1100) 

0.930 

(0020) 

0.917 

(1020) 

0.907 

(0011) 

0.900 

(QUO) 

0.900 

(1011) 

0.890 

(1110) 

0.890 

(0101) 

0.383 

(1101) 

0.873 

(0120) 

0.362 

(0021) 

0.362 

(1021) 

0.352 

(1120) 

0.352 

(0111) 

0.346 

(1111) 

0.336 

(0121) 

0.809 

(1121) 

0.300 
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\'aliie8  of  RlB’  Serond,  Rig  provides  a  lower  bound  for  R  under  much  weaker  assumptions 
about  the  dependence  of  the  test  outcomes. 

Following  [4],  suppose  that  a  device  is  more  likely  to  activate  after  exposure  to  an 
environment  if  it  has  activated  after  expostire  to  a  previous  environment.  Specifically, 
suppose 

PiE,\EinE2)  >  P{E,\Ei) 

PiEilEiDE^nE^)  >  P{E,\Ei). 

then 

R  =  P{Ei)P(E2\Ei)P{E;\EinEi)P(Ei\EinEinE3) 

>  PmPiEm)P{Es\Ei)P{E,\Ei). 


Thus,  Rib  given  in  (2)  provides  a  lower  bound  for  the  MLE  of  R  under  these  much  more 
plausible  assumptions. 

2.4  Lower  Confidence  Bounds 

Also  given  in  Table  2  are  approximate  %'/v  lower  confidence  bounds  found  by  bootstrap¬ 
ping  [3]..  For  each  of  the  24  realizations  under  consideration.  1000  bootstrap  samples  were 
generated  as  follows.  T.et  rik,  i  =  ^ _ _  4.  it  —  1 ..... 24  represent  the  \alue  of  R,  comput¬ 

ed  for  the  realization.  Then,  for  the  k*^  realization,  the  bootstrap  samples  consist  of 
generating  1000  independent  realizations  of  Xi,k  •  ■  • .  where  X^k  •  •  • .  Xi^k  are  indepen¬ 
dent  and  X,,k  ~  Binomi(d{n„  r^)-  From  each  of  the  1000  bootstrap  samples,  the  MLE  of 
R  is  computed,  and  the  lower  confidence  bound  is  taken  as  the  percentile  of  the 
empirical  distribution  of  these  estimates  of  R.. 


13 


The  lower  confidence  bound  for  configuration  (1, 1, 2, 1)  is  slightly  higher  than  the  lower 
confidence  bound  of  [4],  These  lower  confidence  bounds  are,  howewr,  approximations  and 
are  possibly  biased.  To  see  how  these  lower  confidence  bounds  perform,  it  is  a  simple 
matter  to  compute  the  probabilities  that  they  cover  the  true  value  of  R  under  prescribed 
dependence  assumptions.  Returning  to  the  conditional  independence  case,  we  can  use  (1)  to 
compute  the.  probability  of  any  sampling  outcome  given  values  of  . . . ,  R4.  For  simplicity, 
take  /?!  =  ...  =  so  that,  under  conditional  independence,  R  =  Rj.  For  R  >  0.9,  the 
confidence  inter\7ils  given  by  the  lower  confidence  bounds  in  Table  2  ATAVAYS  cowr  R  if 
there  is  at  least  one  failure.  Thus,  the  true  level  of  confidence  for  these  bounds  is  precisely 
the  probability  that  at  least  one  failure  occurs  during  lot  acceptance  testing.  1  -  o  =  1  -  i?®*. 
(n  =  92  is  the  number  of  tested  units.)  Thus,  for  R  =  0.99.  the  confidence  level  for  the 
lower  bound  is  approximately  60%.  and  decreases  to  37%  when  R  increases  to  0.993,; 

This  strange  effect  is  due  to  the  design  of  the  lot  acceptance  plan  <  The  plan  is  designed 
to  ensure  with  95%  confidence  that  a  lot  is  accepted  if  min(Ri,. . . .  R^)  >  0.99.  With  higher 
reliabilities,  there  is  a  very  good  chance  that  there  won’t  be  any  failures  during  the  test, 
and  we  are  powerless  to  produce  an  nccumte  95%  lower  confidence  bound  for  R. 

In  this  section,  we  have  attempted  to  produce  an  accurate  point  estimate  and  lower 
confidence  bound  for  /?,  with  the  ultimate  goal  of  establishing  thresholds  for  assigning 
bonuses.  In  the  next  section,  we  reapproach  the  problem  of  assigning  bonuses. 

3  Alternative  Formulation 

In  this  section,  we  take  a  fresh  approach  to  the  problem  of  awarding  bonuses.  Our  decision 
will  be  of  the  form,  award  a  bonus  if 
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A(X)  >  c 

where  ff(X)  is  a  statistic,  possibly  an  estimate  of  reliablility,  based  on  the  number  of 
devices  that  activate  in  tests  1-4,  X  =  (Xi, ....  X4).  As  we  have  stipulated  in  the  previous 
section,  for  ^(X)  to  be  reasonable,  it  mast  be  nondecreasing  in  each  of  its  arguements,  thus 

if  a:,'  >  j/„  t  =  1, . . . ,  4  then  R{x)  >  i?(y)  where  x  =  (jj . X4)  and  y  =  (yi, ....  y*).  In 

the  previous  section,  Rlb  has  this  property.. 

The  ultimate  goal  of  the  bonus  program  is  to  reward  manufacturers  who  provide  goods 
which  are  more  useful  in  operations  than  the  government-specified  level  of  acceptability. 
Thus,  the  bonus  rule  will  be  based  on  the  goal  of  rewarding  all  manufacturers  for  which 

miv{R] . W4)  >  r  wHth  probability  at  !ea.st  1  -  n.  The  derision  rule  which  accomplishes 

this  is  to  award  bonuses  if 

R(X)  >  c’ 

where  c*  is  the  largest  value  of  c  for  which 

1  -  a  <  P(^(X)  >  c)  (3) 

for  /?i  =  . . .  =  Ri  =  i\ 

As  an  example,  take  R{X)  to  assign  numbers  1. ....  24  to  the  outcomes  leading  to  lot 
acceptance  as  listed  in  Table  4.  Table  4  gives  the  cumulative  probabilities  P(ff(X)  >  c) 
when  /?i  =  . . .  =  /?4  =  r  for  different  \nlu*>s  of  r..  Thus,  if  bontises  are  to  be  aw'ard- 

ed  to  manufacturers  for  which  min{Ri . R.\)  >  0.99ii  with  9,*)%  chance  of  identifying 

these  vendors  based  on  a  lot  acceptance  sampling  plan,  then  the  threshold  score  c*  =  17. 
Here,  bonuses  are  given  to  manufacturers  which  have  one  or  zero  total  failures,  or  have 
(0.0. 1. 1).(0. 1,0. 1).  or  (0. 1. 1.0)  failures.  Note  that  in  this  example  the  values  were  arbi¬ 
trarily  assigned  and  don’t  take  into  account  the  fact  that  test  3  uses  a  larger  sample  of  32 

1 .5 


while  the  other  tests  are  based  on  a  sample  size  of  20. 

Depending  on  the  environment  that  the  device  will  be  used  in,  the  reliability  is  prob¬ 
ably  closer  to  a  weighted  average  of  /?i.  Because  these  weights  differ  for  differ¬ 

ent  types  of  pyrotechnic  devices  and  the  weights  themselves  must  be  estimated  based  on 
the  mission  the  devices  are  used  in,  a  simple  and  conserv'ative  alternative  is  to  consider 

The  minimum  also  has  the  attractive  feature  that  it’s  MLE 
can  be  found  from  the  lot  acceptance  data  without  assuming  anything  about  the  depen¬ 
dencies  between  E\ . E\.  Table  5  gives  the  MLE  km%n  —  Tnw{ki,...,R4)  along  with 

P{kmtn  ^  c)  when  iZi  =  ...;=  =  r  for  various  values  of  r. 

To  see  that  this  rule  acheives  the  desired  goal,  in  otherwords  that 

1  -  o  <  P{k{X)  >  c*) 

for  fimin  ^  f  where  c*  is  chosen  as  in  (3).  and  k{X)  is  a  nondecreasing  function  of 
.Yi . Xi,  we  need  the  notion  of  stochastic  ordering.  The  random  variable  X  is  stochas- 

Ht 

tically  greater  than  the  random  liable  Y.  denoted  .Y  >  Y.  if  P{X  >  x)  >  P{Y  >  x)  for 
all  X.  In  the  multiwiate  exten.sion  of  this  notion,  the  random  vector  X  is  stochastically 
greater  than  the  random  vector  Y  if 

E[h{X)\  >  E[h{Y)] 

for  all  real- valued  functions  h  which  are  nondecreasing  in  their  arguements  for  which 
the  expectation  exists. 

Now  let  A'i,...,X4  be  independent  with  X,  Biuomial{ni, R,)  and  VI,...,  V4  be  in- 

st 

dependent  with  Y,  ~  Binomial{nt,  r),  where  R,  >  r..  Then  X,  >  V,’  for  t  =  1, . . . ,  4  and  by 
Theorem  4.13  of  [1] 

X  =  (.Y, . .Y4)>Y  =  (y,' . ii). 

It) 


Taible  4:  Cumulative  probabilities  for  R(X)  when  Ri  »  r,.,.,Ri  s  r,  where  (iij'tJb,/) 
repreflent  the  number  of  failures  in  tests  1-4  respectively. 

P{k{X)  ^  C) 


e 

(vj»A.O 

R-.99 

R-.995 

R**.999 

24 

(0000) 

0.401 

6.334 

0.912 

23 

(0001) 

0.432 

0.397 

0.991' 

22 

(0010) 

0.310 

0.800 

0.930 

21 

(0100) 

0.891 

0.833 

0.979 

20 

(1000) 

0.772 

0.923 

0.997 

19 

(0011) 

0.793 

0.933 

0.998 

13 

(0101) 

0.'314 

0.943 

0.998 

17 

(0110) 

0.340 

0.953 

0.999 

13 

(1001) 

0.3S3 

0.959 

1.000 

13 

(1010) 

‘0.332 

0.970 

1.000 

14 

(1100) 

0.399 

0.973 

1.000 

13 

(0020) 

0.919 

0.984 

1.000 

12 

(0111) 

0.924 

0.985 

1.000 

11 

(1011) 

0.929 

0.933 

1.000 

10 

(1101) 

0.932 

0.937 

1.000 

9 

(1110) 

0.933 

0.983 

1.000 

3 

(0021) 

0.942 

0.989 

1.000 

7 

(0121) 

0.942 

0.933 

1.000 

3 

(1020) 

0.947 

0.989 

1.000 

5 

(1111) 

0.943 

0.989 

1.000 

4 

(0121) 

0.943 

0.989 

1.000 

3 

(1021) 

0.950 

0.990 

1.000 

2 

(1120) 

0.950 

0.990 

1.000 

1 

(U21) 

0.950 

0.990 

1.000 

j 
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I  .V. 


Table  5:  Cumulative  probabilities  for  Rmm  when  Ri  ^  r,  ...R^  ^  r,  where 
represent,  the  number  of  failiiree  in  tests  %4  respect.ive)y. 


e* 

Mk,i) 

Rs>.99 

c) 

R».995 

R^.999 

1.000 

(0000) 

0.401 

0.634 

0.913 

&.989 

(1000) 

0.482 

0.697 

0.931 

0.969 

1 

(0010) 

(1010) 

(0110) 

0.636 

0.809 

0.961 

0.960 

0.667 

0.319 

0.962 

0.980 

(0001) 

0.918 

0.980 

1.000 

0.938 

(0100) 

(0011) 

(0101) 

(1001) 

(1100) 

(0111) 

(1011) 

(1101) 

(1110) 

(0020) 

0.952 

0.990 

1.000 

(0021) 

(0121) 

(1020) 

(1111) 

(0121) 

(1021) 

(1121) 

(1121) 

0.950 

0.990 

1.000 
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Tliis  gives  us  R{\)  >  ft(Y)  so  that 
P(fi(X)  >  c%  >  PiR(y)  >  c*)  >  1  -  «• 

More  formally,  the  procedure  to  award  bonuses  tests  the  null  hypothesis  Hp  Ri  > 
r, . . . ,  /?4  >  r.  Defining  the  null  hypothesis  in  this  way  alows  us  to  control  the  level  of 
significance  a,  i.e.  the  prohabilHy  of  not  awarding  a  bonus  when  one  is  deserved.  The 
likelihood  ratio  test  (e.g.  [2])  for  this  problem  is  to  award  a  bonus  (i.e.  fail  to  reject  Ho)  if 
^(X)  >  c*  where  R{X)  is  given  by 

-  - Xi)ln{^^)]l{Ri  <  r) 

Hi'  1  —  Hi 

where  !{A)  is  the  indicator  function  of  the  set  d. 

4  Conclusion 

There  are  several  factors  that  make  using  acceptance  test  data  to  award  bonuses  challenging. 
They  all  stem  from  the  fact  that  the  data  is  gathered  from  a  lot  acceptance  sampling  plan 
which  was  not  designed  to  estimate  an  overall  reliability,  nor  was  it  designed  to  give  good 
estimates  of  reliabilities  when  they  are  higher  than  the  minimal  lot  acceptance  standards. 

To  use  the  data  from  the  lot  acceptance  sampling  plan,  the  most  reasonable  approadi 
seems  to  use  Rmm  t^ttd  its  distribution  for  R\  =  ...  =  Ri  to  decide  whether  to  award  a 
bonus. 

There  are  several  ways  that  this  decision  rule  can  be  improved.  These  initiatives  were 
mentioned  earlier,  and  include 

1.  Take  larger  samples  or  sample  sequentially.  This  will  reduce  the  chances  of  awarding 
a  bonus  to  underser\ing  manufacturers.. 
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2.  Construct  a  better  measure  of  overall  reliability.  A  viable  option  is  a  weighted  average 

4 

jrrl 

when*  the  weights  represent  the  proportion  of  the  devices  in  the  field  that  are 
subjected  to  environment  t.  This  would  involve  either  constructing  or  estimating 
different  weights  for  different  types  of  devices. 

3,  Use  R  =  P{Ei  n  £2  0  £3  n  £4)  as  the  measure  of  overall  reliability,  but  collect  data  to 
estimate  the  interaction  between  the  environmental  teats.  R  could  then  He  estimated 
using  a  loglinear  model  based  on  censoretl  observations., 
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